Apparatus and method for displaying image in virtual space

ABSTRACT

An information processing apparatus includes a viewpoint information acquisition unit that acquires viewpoint information concerning a viewer, a data input unit that inputs virtual space data, a specifying unit that specifies a virtual reference surface corresponding to a real reference surface in the virtual space data, a determination unit that determines positional relationship between the virtual reference surface and the real reference surface, a correction unit that corrects the virtual space data based on the positional relationship, and a generation unit that generates a display image based on the corrected virtual space data and the viewpoint information.

BACKGROUND

Field

The present disclosure relates to displaying of a virtual object in avirtual space.

Description of the Related Art

In recent years, studies on mixed reality (MR) for achieving seamlessconnection between a real space and a virtual space have been vigorouslyconducted. For example, a device that presents MR includes the followingconfiguration. Such a device displays an image on which an image of avirtual reality space, e.g., a virtual object or character informationrendered with computer graphics, is overlapped on an image in a realspace captured by an image capturing apparatus such as a video camera.The image of the virtual reality space is generated in accordance with aposition and an orientation of the image capturing apparatus. Thedisplay method by such a device is known as a video see-through method.Such a system can be achieved with a head-mounted display (HMD) used asa display, e.g., see Japanese Patent Application Laid-Open No.2008-134161.

An image display device can be implemented by an optical see-throughmethod. In this method, an image in a virtual space, generated inaccordance with a position and an orientation of a viewpoint of anoperator, is displayed on the HMD displaying an image in the real spacein a see-through manner.

A case is considered in which a viewer experiences MR in a virtualobject such as a building in a mixed reality space. In such a case,alignment of a floor surface of the virtual object and a floor surfacein the real space, on which the viewer is actually standing, has beenconventionally conducted by moving positional coordinates of the virtualobject in accordance with numerical values input with an input devicesuch as a keyboard. Alternatively, the movement is achieved by a manualoperation on a game controller and the like for moving the virtualobject.

SUMMARY

Embodiments are directed to an easy alignment between a floor surface ina real space and a floor surface in a virtual space.

An information processing apparatus according to aspects of embodimentsincludes a viewpoint information acquisition unit configured to acquireviewpoint information concerning a viewer, a data input unit configuredto input virtual space data, a specifying unit configured to specify avirtual reference surface corresponding to a real reference surface inthe virtual space data, a determination unit configured to determinepositional relationship between the virtual reference surface and thereal reference surface, a correction unit configured to correct thevirtual space data based on the positional relationship, and ageneration unit configured to generate a display image based on thecorrected virtual space data and the viewpoint information.

Further features will become apparent from the following description ofexemplary embodiments with reference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a functional configuration of asystem according to a first exemplary embodiment.

FIG. 2 is a diagram schematically illustrating a virtual space.

FIG. 3 is a diagram illustrating a state where a viewer is in a realspace.

FIG. 4 is a diagram illustrating a state where the viewer isexperiencing mixed reality (MR) in an MR space.

FIG. 5 is a diagram illustrating a state where a floor surface in avirtual space is aligned with a floor surface in the real space in theMR space.

FIG. 6 is a flowchart illustrating a procedure of processing accordingto the first exemplary embodiment.

FIG. 7 is a diagram illustrating a method for selecting a floor surface.

FIG. 8 is a block diagram illustrating a functional configuration of asystem according to a second exemplary embodiment.

FIG. 9 is a diagram schematically illustrating a floor surfaceregistration screen.

FIG. 10 is a diagram illustrating how the floor surface is selected andmoved.

FIG. 11 is a flowchart illustrating a procedure of processing accordingto the second exemplary embodiment.

FIG. 12 is a block diagram illustrating a hardware configuration of thesystem according to the exemplary embodiments.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments are described in detail below with reference tothe attached drawings. It is to be noted that the exemplary embodimentsare not limiting.

FIG. 1 is a block diagram illustrating a functional configuration of asystem according to a first exemplary embodiment. As illustrated in FIG.1, the system according to the present exemplary embodiment includes aninformation processing apparatus 1000, an input unit 1100, a displaydevice 1300, and a head-mounted display (HMD) 1200. The informationprocessing apparatus 1000 includes a captured image acquisition unit1010, a viewpoint information measurement unit 1020, a floor surfaceobtaining unit 1030, a floor surface selection unit 1040, a floorsurface position setting unit 1050, a data storage unit 1060, a virtualspace generation unit 1070, an image generation unit 1080, and an imageoutput unit 1090. The information processing apparatus 1000 and the HMD1200 are connected to each other in such a manner that datacommunications can be performed therebetween. The information processingapparatus 1000 and the HMD 1200 can be in wired or wireless connection.

First, the information processing apparatus 1000 will be described.

The captured image acquisition unit 1010 acquires a real space image forthe right eye and a real space image for the left eye, transmitted froman image capturing apparatus 1220 including an image capturing apparatus1220R for the right eye and an image capturing apparatus 1220L for theleft eye, to input an image. The captured image acquisition unit 1010stores the acquired real space images in the data storage unit 1060.

The viewpoint information measurement unit 1020 measures the positionand the orientation of the image capturing apparatus 1220 from thecaptured image stored in the data storage unit 1060 as viewpointinformation. Alternatively, the center of the viewer's head or thecenter between the image capturing apparatus 1220L and the imagecapturing apparatus 1220R can be measured as the viewpoint information,instead of the position and the orientation of the image capturingapparatus 1220. Various research reports related to a method formeasuring viewpoint position and orientation information have beenreleased. For example, see “A Review of Registration Techniques in MixedReality” by Sato, Uchiyama, and Tamura, Transaction of the VirtualReality Society of Japan VOL. 8, NO. 2, pp. 171 to 180, 2003. Herein,any measurement technique can be employed, and a magnetic sensor and anoptical sensor can be used. The measured viewpoint position andorientation information is stored in the data storage unit 1060.

The floor surface obtaining unit 1030 obtains a surface corresponding tothe floor surface in a first virtual object stored in the data storageunit 1060, and stores the floor surface information in the data storageunit 1060. How the floor surface is obtained is described in detail withreference to a flowchart in FIG. 6.

The floor surface selection unit 1040 selects a specific floor surfacefrom the floor surface information stored in the data storage unit 1060,and stores the information about the floor surface in the data storageunit 1060. How the floor surface is selected is described in detailbelow with reference to the flowchart in FIG. 6.

The floor surface position setting unit 1050 performs setting in such amanner that a height position of the floor surface information stored inthe data storage unit 1060 matches the position of the floor in the realspace on which the viewer stands. How the setting is performed isdescribed in detail below with reference to the flowchart in FIG. 6.

The virtual space generation unit 1070 reads and inputs data on avirtual space stored in the data storage unit 1060, and generates thevirtual space. The data on the virtual space includes data related tovirtual objects forming the virtual space and data related to a lightsource that illuminates the inside in the virtual space.

The image generation unit 1080 generates an image (virtual space image)in the virtual space as viewed from the viewpoint in the virtual spacegenerated by the virtual space generation unit 1070. Then, the imagegeneration unit 1080 first renders the real space image, stored in thedata storage unit 1060, in a memory managed by the image generation unit1080. Then, the image generation unit 1080 renders the virtual spaceimage on the rendered real space image in a superimposed manner. Thus, acomposite image from the real space image and the virtual space image isgenerated in the memory. Then, the image generation unit 1080 outputsthe generated virtual reality space image to the display devices 1210Rand 1210L of the HMD 1200. At the same time, the virtual space image canbe displayed on the display device 1300.

Alternatively, a display device other than the HMD 1200 can be used.Examples of such a display device include display terminals such as atablet or a smartphone. Techniques for generating an image in thevirtual space as viewed from a viewpoint with a predetermined positionand orientation have been known, and thus will not be described indetail.

The input unit 1100 is a device with which a user operation can be inputto the information processing apparatus 1000. The input unit 1100 can beused when the information processing apparatus 1000 issues aninstruction to the floor surface selection unit 1040.

As described above, the data storage unit 1060 stores various types ofinformation, and includes a random-access memory (RAM) and a hard diskdrive (HDD) device. The data storage unit 1060 stores the information tobe stored in the data storage unit 1060 as described above, as well asinformation described as known information in the present exemplaryembodiment.

Next, the HMD 1200 will be described. The HMD 1200 is a displayapparatus to which the display device 1210 including a liquid crystalscreen and the image capturing apparatus 1220, such as a video camera,are fixed. Unless the display device 1210R for the right eye, thedisplay device 1210L for the left eye, the image capturing apparatus1220R for the right eye and the image capturing apparatus 1220L for theleft eye are specifically described, hereinafter, a display device andan image capturing apparatus are simply referred to as the displaydevice 1210 and the image capturing apparatus 1220, with no referencesigns R and L, respectively.

FIG. 2 is diagram schematically illustrating a virtual space. Virtualspace 2601 includes a virtual light source 2210, a virtual object 2502,a virtual object 3100, a virtual surface 2504, a virtual surface 2503, avirtual surface 3400, and a virtual surface 2501. Each number of virtuallight sources, virtual objects and virtual surfaces can be one or more.The virtual light source 2210 can be data related to a light source thatilluminates the inside in the virtual space 2601, and can be an imagebased lighting (IBL).

The virtual object 2502, a virtual object 2510, a virtual object 2511, avirtual object 2512, and a virtual object 3100 can each be a surfacemodel or a solid model. A front surface of the virtual surface 2504 is asurface in a normal direction 2602. Similarly, a front surface of thevirtual surface 2503 is a surface in a normal direction 2603. The sameapplies to other surfaces.

FIG. 6 is a flowchart illustrating a procedure of processing executed bythe information processing apparatus 1000 to generate a mixed reality(MR) image, and output the MR image to the HMD 1200 or the displaydevice 1300.

First, in step S601, the captured image acquisition unit 1010 acquiresthe captured image from the image capturing apparatus 1220 illustratedin FIG. 1. The viewpoint information measurement unit 1020 measuresposition and orientation information on the image capturing apparatus1220 based on the captured image, to measure viewpoint information.Various research reports related to a method for measuring the positionand the orientation have been released, e.g., “A Review of RegistrationTechniques in Mixed Reality” as described above. In the presentexemplary embodiment, any known measurement technique can be employed,and a magnetic sensor and an optical sensor can be used.

Next, in step S602, the floor surface selection unit 1040 selects afloor surface from the virtual space information. A method for selectingthe floor surface is described with reference to FIG. 2.

In FIG. 2, the floor surface with the normal direction extending upward(in the positive direction on a Z axis in the present exemplaryembodiment) is selected in the virtual space 2601. The surface 3400 withthe normal direction extending upward (in the positive direction on theZ axis) is one example of the floor surface in the virtual space 2601.Similarly, other surfaces satisfying the condition are automaticallyselected.

Alternatively, the floor surface can be selected as follows.Specifically, any number can be set as the number of vertices of asurface, and a surface with vertices of the set number or more can beselected as the floor surface. Furthermore, any value can be set for thearea of a surface, and a surface with an area larger than the set valuecan be selected as the floor surface.

Another method of selecting the floor surface is described withreference to FIG. 7 illustrating an example in a virtual space. In themethod illustrated in FIG. 7, the floor surface is selected based on adirection of a normal line of a surface and a distance between surfaces.A surface 7001 includes a normal line 7002. A surface 7008 includes anormal line 7007. A surface 7009 includes a normal line 7006. A surface7010 includes a normal line 7005. A distance 7003 is a distance betweenthe surface 7001 and the surface 7009. A distance 7004 is a distancebetween the surface 7001 and the surface 7010.

In this example, the method of selecting the floor surface includesacquiring a distance between a surface with a normal line extendingvertically downward (in the negative direction on the Z axis in thepresent exemplary embodiment) and a surface with a normal line extendingvertically upward (in the positive direction on the Z axis in thepresent exemplary embodiment). For example, the surface having thenormal line extending vertically downward is, for example, the surface7001, and the surface with the normal line extending vertically upwardis, for example, the surface 7005, the surface 7008, the surface 7009,and the surface 7010. Only a surface having the normal line extendingvertically upward and having a distance that is longer than or equal toa predetermined distance (a distance 7011 or more) is selected as thefloor surface.

Thus, the surface 7009 with the distance that is longer than or equal tothe distance 7011 is selected as the floor surface. Similarly, surfaceshaving a normal line extending upward (in the positive direction on theZ axis in the present exemplary embodiment) and having the distance thatis longer than or equal to the distance 7011, in the virtual space, areselected as the floor surfaces. A surface having a normal line extendingvertically upward, from among surfaces having distances to a closestsurface having a normal line extending in the opposite direction(vertically downward) being longer than or equal to the distance 7011,can be selected as the floor surface.

In step S603, the floor surface position setting unit 1050 aligns thefloor surface in the virtual space with the floor surface in the realspace. How the alignment is performed will be described with referenceto FIG. 3 to FIG. 5.

FIG. 3 illustrates a state where a viewer 2000 stands in a real space.In FIG. 3, the portions that are the same as those in FIG. 2 are denotedwith the same reference numerals, and the descriptions thereof will beomitted. A height 2100 is a height from the floor surface to a positionof the viewer's eyes. The height 2100 is obtained by using the positionand orientation information obtained in step S601 as described above. Areal floor surface 2200 is an actually existing floor surface on whichthe viewer stands. A real object 2300 is an actually existing object.

FIG. 4 illustrates a state where the viewer is experiencing MR in the MRspace. Portions that are the same between FIG. 2 and FIG. 3 are denotedwith the same reference numerals, and the description thereof will beomitted. A distance 3200 is a distance between a virtual floor surface3400 and a viewpoint of the viewer 2000. A distance 3300 is a distancebetween the real floor surface 2200 and the virtual floor surface 3400.

FIG. 5 illustrates a state where the floor surface in the virtual spaceis aligned with the floor surface in the real space in the mixed realityspace. Portions in FIG. 5 that are the same as those in FIGS. 2 to 4 aredenoted with the same reference numerals, and the description thereofwill be omitted. A virtual object 4100 corresponds to the virtual object3100, illustrated in FIG. 3, after being moved. A virtual floor surface4400 corresponds to the virtual floor surface 3400, illustrated in FIG.4, after being moved.

Next, a method of aligning the floor surface in the virtual space withthe floor surface in the real space is described. When the viewerexperiences MR, the experience generally starts with the body of theviewer penetrating the virtual floor surface as illustrated in FIG. 4,unless the coordinate value of the virtual floor surface in the Z axisdirection is not 0.

Then, the virtual floor surface 3400 having the distance 3200 to theviewpoint of the viewer 2000 is extracted. The distance 3200 is thesmallest one of the distances from the floor surfaces selected in stepS602 to the viewpoint. Then, the distance 3300 between the virtual floorsurface 3400 and the real floor surface 2200 is obtained, and thevirtual space 2601 illustrated in FIG. 2 is moved by the distance 3300.As a result, the virtual floor surface 3400 and the virtual object 3100respectively move to the positions of the virtual floor surface 4400 andthe virtual object 4100 in FIG. 5. Similarly, the virtual objects, thevirtual surfaces, and the virtual light source, as the components in thevirtual space 2601, are moved.

In the present exemplary embodiment, the movement distance 3300 isobtained by extracting the floor surface having the shortest distance3200 to the viewpoint of the viewer 2000. Alternatively, the viewer canuse a graphic user interface (GUI) to designate a desired floor surfacefrom a plurality of floor surfaces obtained in step S602.

In step S604, the floor surface position setting unit 1050 sets theposition and orientation information of a virtual camera, based on theposition and orientation information of the image capturing apparatus1220 obtained in step S601, and places the virtual camera in the virtualspace after being moved in step S603. Then, the virtual space generationunit 1070 generates a virtual image based on the position andorientation information of the image capturing apparatus 1220 obtainedin step S601 and the information in the virtual space after the movementin step S603. In step S605, the image generation unit 1080 generates animage obtained by overlaying the virtual image on the captured imageacquired in step S601.

FIG. 8 is a block diagram illustrating an example of a functionalconfiguration of a system according to a second exemplary embodiment. Aconfiguration that is the same as that in FIG. 1 is not described. Asillustrated in FIG. 8, the system according to the present exemplaryembodiment includes an information processing apparatus 8000, the inputunit 1100, the display device 1300, and the HMD 1200. The informationprocessing apparatus 8000 includes the captured image acquisition unit1010, the viewpoint information measurement unit 1020, a floor surfaceregistration unit 8030, a floor surface selection unit 8040, the floorsurface position setting unit 1050, the data storage unit 1060, thevirtual space generation unit 1070, the image generation unit 1080, andthe image output unit 1090. The information processing apparatus 8000and the HMD 1200 are connected so as to enable data communicationstherebetween. The connection can be conducted in a wired or wirelessmanner.

First, the information processing apparatus 8000 will be described. Thecaptured image acquisition unit 1010, the viewpoint informationmeasurement unit 1020, the floor surface position setting unit 1050, thedata storage unit 1060, the virtual space generation unit 1070, theimage generation unit 1080, and the image output unit 1090 are similarto those illustrated in FIG. 1.

The floor surface registration unit 8030 stores data concerning a floorsurface input from the input unit 1100 in a data storage unit 1060, aswill be described in detail with reference to FIG. 9.

FIG. 9 is a diagram schematically illustrating a floor surfaceregistration screen 9000 used by the floor surface registration unit8030 for the registration. A name input field 9100 is a field where aname is input and displayed, based on information input by the viewerthrough the input unit 1100. This information is stored in the datastorage unit 1060. A height input field 9200 is a field for inputtingand displaying height information based on information input by theviewer through the input unit 1100. The information is stored in thedata storage unit 1060.

The floor surface selection unit 8040 selects a predetermined floorsurface from a plurality of floor surface candidates stored in the datastorage unit 1060, and stores the information of the selected floorsurface in the data storage unit 1060. A method of selecting the floorsurface is described in detail with reference to a flowchart in FIG. 11.

The floor surface position setting unit 1050 sets the height position ofthe floor surface in the virtual space based on the floor surfaceinformation that is stored in the data storage unit 1060 and correspondsto the floor surface selected by the floor surface selection unit 8040.A setting method is described in detail below with reference to theflowchart in FIG. 11.

FIG. 11 is a flowchart illustrating a procedure of processing executedby the information processing apparatus 8000. The processing is executedfor generating an MR image after the floor surface information has beenregistered by the registration unit 8030, and outputting the MR image tothe HMD 1200 or the display device 1300. Processing that is the same asthat in FIG. 6 will not be described. FIG. 10 illustrates how a floorsurface is selected in step S1102 and how the floor surface is moved instep S1103. First, FIG. 10 is described, and then each step in FIG. 11is described.

Floor surface information registered in advance by the floor surfaceregistration unit 8030 is displayed in a selectable manner in a GUI12600. In the illustrated example, floor numbers and heights of thefloor surfaces are displayed. The GUI 12600 can be displayed on avirtual space or can be displayed in other forms, such as a dialog. Afloor surface selection field 12500 indicates a floor surface selectedfrom the information registered in the floor surface registration unit8030, based on an event input by the viewer through the input unit 1100.

The viewer 1200 is a simple representation of the person experiencingthe MR. A floor surface 12200 is a floor surface of the real space. Avirtual floor surface 12400 is a floor surface existing in the virtualspace. In the present exemplary embodiment, a distance 12300 correspondsto a height (e.g., 1000 mm in the present exemplary embodiment)registered in the floor surface registration unit 8030.

In step S1102, the floor surface selection unit 8040 selects a floorsurface from a plurality of pieces of floor surface information on theGUI 12600, based on an operation performed on the input unit 1100. Instep S1103, the surface floor position setting unit 1050 moves the floorsurface 12400 in the virtual space toward the floor surface 12200 in thereal space by the distance 12300, based on the floor surface informationselected in step S1102, so as to match the height therebetween. In stepsS1104 and S1105, the virtual space generation unit 1070 generates thevirtual space and the image generating unit 1080 generates an imageobtained by overlaying the virtual image on the actual image as in stepsS604 and S605.

In the exemplary embodiments described above, a case is described inwhich the floor surface in the virtual space is aligned with the floorsurface in the real space when the image in the virtual space isdisplayed on the image in the real space in an overlapping manner. Thefloor surface in the virtual space can also be aligned with the floorsurface in the real space when only the image in the virtual space isdisplayed. Thus, the viewer viewing the image in the virtual space canview an image with a little uncomfortable feeling compared with the realspace where the viewer is positioned.

In the exemplary embodiments described above, a case is described wherethe floor surface in the virtual space is aligned with the floor surfacein the real space. Alternatively, a surface other than the floor surfacecan be a reference surface for the alignment. For example, a surface ofa workbench in the virtual space can be aligned with a surface of aworkbench in the real space.

The components of the information processing system illustrated in FIG.1 are described as being implemented by hardware in the exemplaryembodiments described above. Alternatively, some of the components canbe implemented by software. In such a case, a computer having theremaining components implemented by hardware executes the software toperform the operations of the information display device described inthe exemplary embodiments described above.

FIG. 12 is a block diagram illustrating a hardware configuration of theinformation processing system according to the exemplary embodiments. Acentral processing unit (CPU) 15001 controls the entire computer byusing a program and data stored in a random-access memory (RAM) 15002and a read-only memory (ROM) 15003, and executes the processingdescribed above, which has been described as being executed by theinformation processing apparatus 1000 in the exemplary embodimentsdescribed above.

The RAM 15002 has an area for temporarily storing a program and dataloaded from an external storage device 15007 and a storage medium drive15008. The RAM 15002 also includes an area for temporarily storing datato be transmitted to the outside through an interface (I/F) 15009. TheRAM 15002 includes a work area used by the CPU 15001 to execute varioustypes of processing. In other words, the RAM 15002 can provide variousareas as appropriate. For example, the RAM 15002 also functions as thedata storage unit 1020 illustrated in FIG. 1.

The ROM 15002 stores setting data and a boot program for the computer. Akeyboard 15004 and a mouse 15005 are examples of an operation inputdevice. The viewer using the computer can issue various instructions tothe CPU 15001 by operating the operation input device.

A display unit 15006 includes a cathode ray tube (CRT) or a liquidcrystal screen, and can display a result of the processing executed bythe CPU 15001 with images and texts.

The external storage device 15007 is a large capacity informationstorage device represented by an HDD device. The external storage device15007 stores an operating system (OS), as well as a program and data forcausing the CPU 15001 to execute the processing described above as beingexecuted by the information processing apparatus. The program includesprograms corresponding to the captured image acquisition unit 1010, theviewpoint information measurement unit 1020, the floor surface obtainingunit 1030, the floor surface selection unit 1040, the floor surfaceposition setting unit 1050, the virtual space generation unit 1070, theimage generation unit 1080, and the image output unit 1090. The dataincludes data concerning MR experience environment information, and theknown information described above.

The program and the data stored in the external storage device 15007 areloaded onto the RAM 15002 as appropriate, in accordance with the controlperformed by the CPU 15001. The CPU 15001 executes processing by usingthe loaded program and data, and thus executes the processing describedas being executed by the information processing apparatus. The externalstorage device 15007 can be used as the data storage unit 1060illustrated in FIG. 1.

The storage medium drive 15008 reads a program and data stored in astorage medium such a compact disc (CD)-ROM or a digital versatile disc(DVD)-ROM, and writes a program and data to such a storage medium. Thestorage medium can store a part of or all the programs and datadescribed as being stored in the external storage device 15007. Theprogram and data, read from the storage medium by the storage mediumdrive 15008, are output to the external storage device 15007 and the RAM15002.

The I/F 15009 includes a digital input/output port such as a universalserial bus (USB) and an Ethernet® port with which an image is output tothe display device 1010. Data received through the I/F 2009 is input tothe RAM 15002 and the external storage device 15007. The componentsdescribed above are connected to each other via a bus 15010.

Other Embodiments

Embodiment(s) can also be realized by a computer of a system orapparatus that reads out and executes computer executable instructions(e.g., one or more programs) recorded on a storage medium (which mayalso be referred to more fully as a ‘non-transitory computer-readablestorage medium’) to perform the functions of one or more of theabove-described embodiment(s) and/or that includes one or more circuits(e.g., application specific integrated circuit (ASIC)) for performingthe functions of one or more of the above-described embodiment(s), andby a method performed by the computer of the system or apparatus by, forexample, reading out and executing the computer executable instructionsfrom the storage medium to perform the functions of one or more of theabove-described embodiment(s) and/or controlling the one or morecircuits to perform the functions of one or more of the above-describedembodiment(s). The computer may comprise one or more processors (e.g.,central processing unit (CPU), micro processing unit (MPU)) and mayinclude a network of separate computers or separate processors to readout and execute the computer executable instructions. The computerexecutable instructions may be provided to the computer, for example,from a network or the storage medium. The storage medium may include,for example, one or more of a hard disk, a random-access memory (RAM), aread only memory (ROM), a storage of distributed computing systems, anoptical disk (such as a compact disc (CD), digital versatile disc (DVD),or Blu-ray Disc (BD)™), a flash memory device, a memory card, and thelike.

While exemplary embodiments have been described, it is to be understoodthat the invention is not limited to the disclosed exemplaryembodiments. The scope of the following claims is to be accorded thebroadest interpretation so as to encompass all such modifications andequivalent structures and functions.

This application claims the benefit of Japanese Patent Application No.2016-032477, filed Feb. 23, 2016, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An information processing apparatus comprising: aviewpoint information acquisition unit configured to acquire viewpointinformation concerning a viewer; a data input unit configured to inputvirtual space data; a specifying unit configured to specify a virtualreference surface corresponding to a real reference surface in thevirtual space data; a determination unit configured to determinepositional relationship between the virtual reference surface and thereal reference surface; a correction unit configured to correct thevirtual space data based on the positional relationship; and ageneration unit configured to generate a display image based on thecorrected virtual space data and the viewpoint information.
 2. Theinformation processing apparatus according to claim 1, furthercomprising: an extraction unit configured to extract at least one bottomsurface from the virtual space data; and a selection unit configured toselect, from the extracted at least one bottom surface, a bottom surfaceas the reference surface in the virtual space.
 3. The informationprocessing apparatus according to claim 2, wherein the selection unit isfurther configured to select, based on a distance between the realreference surface and the extracted at least one bottom surface, thebottom surface as the reference surface in the virtual space.
 4. Theinformation processing apparatus according to claim 2, wherein theselection unit is further configured to select, based on number ofvertices of each of the extracted at least one bottom surface, thebottom surface as the reference surface in the virtual space.
 5. Theinformation processing apparatus according to claim 2, wherein theselection unit is further configured to select, based on an area of eachof the extracted at least one bottom surface, the bottom surface as thereference surface in the virtual space.
 6. The information processingapparatus according to claim 2, wherein the extraction unit is furtherconfigured to extract a surface with a normal line extending verticallyupward as the bottom surface.
 7. The information processing apparatusaccording to claim 6, wherein the extraction unit is further configuredto extract, as the bottom surface, a surface with the normal lineextending vertically upward and having a distance longer or equal to apredetermined length from a closest one of surfaces with the normal lineextending vertically downward.
 8. The information processing apparatusaccording to claim 1, further comprising: a presenting unit configuredto present a plurality of candidates of the reference surface in thevirtual space; and a selection unit configured to select, as thereference surface in the virtual space, a candidate selected from theplurality of candidates.
 9. The information processing apparatusaccording to claim 8, wherein the presenting unit is further configuredto present a plurality of floor numbers as the plurality of candidates.10. The information processing apparatus according to claim 8, whereinthe presenting unit is further configured to present a plurality ofheights as the plurality of candidates.
 11. The information processingapparatus according to claim 1, wherein the viewpoint informationacquisition unit is configured to acquire the viewpoint informationbased on an image captured by an image capturing unit.
 12. Theinformation processing apparatus according to claim 1, wherein theviewpoint information includes a position of a viewpoint of the viewer.13. The information processing apparatus according to claim 1, whereinthe viewpoint information includes an orientation of a viewpoint of theviewer.
 14. The information processing apparatus according to claim 1,wherein the reference surface is a floor surface.
 15. The informationprocessing apparatus according to claim 1, wherein the correction unitis further configured to shift data on the virtual space based on adistance between the reference surface in the virtual space and thereference surface in the real space.
 16. The information processingapparatus according to claim 1, further comprising an image input unitconfigured to input a captured image in the real space, and wherein thegeneration unit is further configured to generate the display image bycombining the corrected virtual space data and the captured image in thereal space.
 17. An information processing method comprising: acquiringviewpoint information concerning a viewer; inputting virtual space data;specifying a virtual reference surface corresponding to a real referencesurface in the virtual space data; determining positional relationshipbetween the virtual reference surface and the real reference surface;correcting the virtual space data based on the positional relationship;and generating a display image based on the corrected virtual space dataand the viewpoint information.
 18. A non-transitory computer-readablestorage medium storing computer-executable instructions for causing acomputer to implement an information processing method, the informationprocessing method comprising: acquiring viewpoint information concerninga viewer; inputting virtual space data; specifying a virtual referencesurface corresponding to a real reference surface in the virtual spacedata; determining positional relationship between the virtual referencesurface and the real reference surface; correcting the virtual spacedata based on the positional relationship; and generating a displayimage based on the corrected virtual space data and the viewpointinformation.